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PREDICTING  PEER  EVALUATIONS  FROM  BIOGRAPHICAL  INFORMATION 


Peer  eveluationa  have  long  been  of  Interest  to  military  behavioral 
science  researchers.  Initially,  concern  was  with  the  validity  and  reli¬ 
ability  of  peer  evaluations  for  use  In  the  assessment  of  leadership  abil¬ 
ity  and  job  proficiency.  Findings  resulted  In  the  adoption  of  peer  ratings 
as  an  Important  element  In  leadership  assessment  In  several  Army  programs, 
most  Important  among  them  being  the  U.S.  Military  Academy  and  Officer 
Candidate  Schools  (OCS) . 

Concurrently,  researchers  began  an  examination  of  Issues  related  to 
bias  In  the  peer  evaluation  process.  These  research  programs  have  dealt 
primarily  with  the  effects  of  race  and  friendship  on  the  evaluation  process. 
Cox  and  Krumboltz  (1958)  Investigated  the  effects  of  racial  bias  on  peer 
ratings  of  leadership  ability.  With  a  sample  comprising  469  tdiltes  and  64 
blacks  distributed  across  9  flight  training  teams  In  the  Air  Force,  they 
examined  the  extent  to  which  the  race  of  a  ratee  affected  the  rating  score 
he  received  from  both  his  black  and  his  white  peers.  In  the  main,  the  data 
supported  the  hypothesis  that  Individuals  of  a  given  race  receive  higher 
mean  ratings  from  members  of  their  own  race  than  from  members  of  the  other 
race.  This  finding,  however,  did  not  hold  for  all  flight  training  teams. 
Over  all  teams,  the  two  races  were  able  to  reach  substantial  agreement 
with  regard  to  the  rank  ordering  of  Individuals  on  leadership  ability 
(r  »  .76).  The  researchers  concluded  that  the  bias  observed  was  In  no 
way  complete,  and  that  for  practical  purposes  the  validity  of  the  peer 
evaluation  process  was  not  violated  by  what  bias  did  exist.  Similar 
conclusions  have  been  reached  by  other  researchers  working  in  both  military 
(DeJung  and  Kaplan,  1962)  and  Industrial  (Schmidt  and  Johnson,  1973) 
settings. 

Hollander  and  Webb  (1955)  collected  peer  ratings  on  leadership,  friend¬ 
ship,  and  followership  from  187  Naval  Aviation  Cadets.  Their  results  led 
them  to  conclude  that  leadership  ratings  were  not  a  function  of  frlendshlo 
ties  when  leadership  was  highly  correlated  with  followership.  The  Indepen¬ 
dence  of  leadership  ratings  from  friendship  ties  was  given  additional 
support  In  subsequent  research  using  OCS  students'  peer  evaluations  of 
leadership  qualities  to  predict  OCS  academic  averages  (Hollander,  1956). 

The  present  research  was  designed  to  broaden  the  simultaneous  examina¬ 
tion  of  potential  bias-producing  variables  by  an  assessment  of  the  relation¬ 
ship  of  selected  biographic  variables  to  peer  evaluation  scores. 


METHOD 

SAMPLE 

Subjects  were  enlisted  members  of  the  27th  Brigade,  42nd  Division  of 
the  New  York  State  National  Guard.  Data  were  collected  at  Fort  Drum, 

New  York,  during  the  annual  two-week  training  period  of  the  27th  Brigade. 
The  participating  units  were  not,  however,  regular  training  units.  The 


companies  chosen  were  rej?ularly  constituted  National  Guard  units  whose 
personnel  were  assluned  on  a  lonK-term  basis.  Five  platoons  weii  selected 
from  the  27th  Brtitade  to  he  representative  of  various  combat  .iiul  combat 
support  units.  Table  1  shows  the  company  and  number  of  Individuals  sampled. 
Since  the  data  collection  was  carried  out  In  the  field  durlntt  breaks  In 
the  tralnlnK  schedule,  situational  factors  made  It  Impossible  for  all  the 
designated  Guardsmen  to  participate  In  the  procedure,  result  Inn  In  a 
reduced  N  of  Td. 


Table  1 

PUITODNS  SEl.F.GTF.D  ANtl  NUMBER  OF 
PART  in  PANTS  FRO»1  EACH  PLATOON 


Platoon  N 

Co.  0,  102nd  Medlc.al  Battalion  Id 

Co.  B,  1 /127th  Armor  Battalion  11 

Headquarters  and  Headquarters  Go. , 

1/lOSth  Infantry  20 

Go.  A,  1/1 74th  Infantry  17 

Gomhat  Support  ('o.  ,  2/108th  Infantry  1 2 

7d 


VARIABLES 

Data  were  collected  uslnn  two  Instruments.  The  Guardsmen  were  first 
directed  to  fill  out  a  hlonraphic  survey  which  tapped  demonranhlc  Infor- 
m.it  Ion  on  their  educational,  vocational,  and  military  backcroiinds .  Tlie 
specific  variables  were:  ane ,  level  of  edticntlon,  military  rank,  lemuh 
of  time  In  the  National  Guard,  lennth  of  time  in  their  nresent  unit,  and 
wliether  a  sqtiad  leader  or  not. 


Immediately  followinit  completion  of  the  biographic  survey,  the  men 
were  (tlven  an  associate  (peer)  evaluation  form.  They  were  iii>t  nictod 
to  choose  from  among  the  other  members  of  their  platoon  the  si.,  individuals 
ihev  considered  best  at  getting  the  Job  done  and  the  six  thc\  ifnsidored 
worst  at  getting  the  Job  done.  Choices  were  recorded  on  opi  leal  .i.-.mning 
answer  sheets.  The  nominations  were  converted  to  a  numerical  score  referred 
to  here  as  the  peer  evaluation  score.' 


RKSITI.TS 


A  set  of  Pearson  product-moment  correlation  coeltl'  ienls  w.ts  c.ilculated 
in  order  to  examine  the  degree  and  direction  of  the  int enelat lonshlp  of 
the  biographic  variables  and  the  relationship  between  c.ich  of  the  biographic 
variables  and  the  peer  evaluation  scores.  The  results  presented  In  Table 
demonstrate  that  the  biographic  variables  are  all  significantly  correlated 
with  the  peer  evaluation  scores.  The  moat  striking  of  l  liese  relationships 
is  between  the  peer  evaluation  score  and  military  rank  (r  •  .oP. 

Table  2 

ZKRO-ORDF.R  INTKRCORRKLATION  M/\TR1X  VOR  bllV.RAPHTC 
■AND  PEFR  FVAHT.ATION  VARIABLI-S 
(N  -  76) 


Ai'.K 

KOl'CA- 

noN 

RANK 

TIME  IN 
C.l'.ARPi 

TIME  IN 
UN  I  r 

1 EAOER 

SHIP 

POSITK 

Educat ion 

.38* 

Rank 

.53* 

.45* 

T  tme  in  C.uard 

.52* 

.26** 

.  50* 

Time  in  I'nlt 

.40* 

-.02 

.  2-4** 

.67* 

l.eadership 

Posl t ion 

.38* 

.23** 

.61* 

.27** 

.06 

Peer  Kvaluatlon 
Score 

.4Q* 

.42* 

.61* 

.43* 

,  •ict* 

.46* 

• •'  I'lii 


'  The  conversion  formula  used  was:  Peer  .Score  “  1(^10  ♦  I  +  '(N  -  "  -  1  -  1'. 

N  -  I 

where  H  -  number  of  high  nominations,  I.  •  number  of  low  nominations,  and 
N  -  number  of  subjects  in  the  peer  rating  group. 


Tha  biographic  vartablaa  wara  found  to  ba  hlf^ly  Intarrelaicd.  lu 
ordar  to  dataralna  if  tha  varloua  biographic  variablaa  were  lndipendfntly 
accounting  for  thair  high  corralatioo  with  tha  paar  evaluation  ai-oris,  a 
aat  of  partial  correlation  coafficianta  waa  coaq^utad  in  which  the  relatlon- 
ahip  batwaan  any  glvan  biographic  variable  and  tha  peer  evaluation  score 
waa  aaaaaaad  by  controlling  for  other  biographic  variables.  Table  3  shows 
that  controlling  for  rank  alone,  or  for  rank  in  coiritlnatlon  with  time  In 
unit,  could  reduce  tha  correlation  of  tha  other  variables  to  nonslgnlficance. 
The  correlation  trlth  rank  could  not  be  reduced  below  r  *  .37  (p  <  .001), 
regardless  of  which  controlling  variables  were  used. 


Table  3 

PARTIAL  CORRELATIOM  KTUBBR  BIOQtAPHIC  VARIABLES 
AMD  PEED  EVALUATION  SCORES 
(M  -  76) 


BIOGRAPHIC 

VARIABLE 

ONITROLLING  FOR 

'^partial 

AGE 

RANK  6  TIME  IN  UNIT 

.15 

EDUCATION 

RANE 

.22 

TIME  IN  GUARD 

RANK 

.19 

TIIC  IN  UNIT 

RANK 

.22 

A  stap-wisa  Bultipla  ragraaaion  procedure  was  used  to  assess  the 
relative  powers  of  tha  biographic  variables  to  predict  peer  evaluation 
scores.  These  results,  presented  in  Table  4,  indicate  that  rank,  time 
in  unit,  and  education  contributad  aignlficantly  to  the  prediction  equation. 
An  analysis  of  the  Increase  in  predictability  that  could  result  from  an 
equation  which  contains  those  three  variables  and  leadership  position  over 
an  aquation  which  includes  only  the  three  forswr  variables  indicates  the 
nonsignificant  contribution  of  leadership  position.  Additional  evidence 
for  the  lack  of  practical  significance  of  leadership  position,  time  iu 
Guard,  and  age  in  predicting  peer  evaluations  is  given  by  the  summarv 

section  of  Table  4  where  "R^  change"  gives  the  Increase  in  total  variaiue 
accounted  for  by  each  variable.  Fron  this  sane  section  of  Table  s,  it  is 
clear  that  rank  is  the  prlnary  contributor  to  the  prediction  equation, 
followed  by  tiaw  in  unit  and  education  as  distant  second  and  third 
contributors. 
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Tahle  4 


RKf.RKSSION  ANALYSIS  m'DlCTlNC  PFKR  KVAU'ATKA 
SCOW;s  FROM  BIOCRAPHIC  VARIABLES 
(N  -  7f.) 


VARIABLES  IN  THE  E()l'ATION 

B 

BETA 

STO  ERROR 

B 

F 

RANK 

17.57 

0.  n 

6.P2 

7.05* 

TIME  IN  ITNIT 

:7.45 

0.46 

'.0  < 

1 5.25** 

EDl’CATION 

S.48 

0.26 

I.IS 

7.2p* 

LKAOERSMIP  POSITION 

21.15 

0.20 

11  .07 

TlMi:  IN  (UIARTl 

10. <58 

0.18 

7.5: 

2.14*** 

•AOE 

0.7‘) 

0.05 

1  .  58 

0 .  J  s  * 

CONSTANT 

97.60 

R  0.74 

R‘  0.‘>5 

STO  ERROR  32.1‘» 

ENTIRI’  EQUATION: 

FCp.pO)  -  n.8B 

(P  .001) 

SU'PLARY 

VARIABLE 

MULTIPLE 

R 

•> 

R- 

R‘ 

CHANCl' 

RANK 

.Pi 

.  37.*5 

.  <'5 

TIME  IN  t'NlT 

.PS 

,  4  6  S  S 

.001 

Em' CAT  I  ON 

.n 

.  5075 

.0..2 

LEAPERSHIP  POSITION 

.  7  1 

.  5  12  1 

.025 

TIME'  IN  (;r,\Kr> 

* 

.545  \ 

.OM 

ACE 

.7.* 

.  54'0 

.002 

*P  .01 

**p  .001 

***  N.S. 


DISCUSSION 


Clven  the  etrong  relationship  between  individuals'  rank  and  il.  i 
peer  evaluation  scores,  the  major  question  was  how  reasonable  it  i:.  tv' 
expect  such  a  relationship.  A  major  premise  ot  the  peer  nonlnatlon  I'l 
rating  process  Is  that  members  ot  the  group  are  able  to  compare  ttte 
prot'lclency  of  group  members  in  the  performance  of  their  lobs.  With  each 
advancement  to  hlghei  rank,  the  individual  essentiallv  performs  i  vlitferent 
Job,  if  not  In  task  c.'ntent ,  then  in  the  responsibilities  associated  with 
his  formal  position.  Therefore,  within  anv  given  rank  there  should  be 
a  distribution  of  peer  scores  such  that  some  Individuals  .ire  i.itcd  "best" 
and  others  "worst"  at  performing  their  Job.  When  this  distribution  does 
not  occur,  one  possible  Inference  is  that  bias  has  been  i nt  i v'ducei!  into 
the  evaluation  process.  In  this  case.  Individuals  of  the  higher  ranks 
were  rated  "best"  at  their  Jobs  and  Individuals  of  the  lower  ranks  were 
rated  "worst"  at  their  Jobs.  It  mav  be  Inferred  that  the  members  of 
the  group  were  not  evaluating  each  individual  based  upon  the  job  behaviors 
associated  with  each  ratee's  actual  Job,  but  rather  that  each  rater  was 
redefining  the  term  "Job"  in  such  a  manner  as  to  arrive  at  one  set  of 
U'b  behaviors  which  Cv'uld  be  applied  comonlv  across  ranks  to  .ill  group 
iiwmbers . 

ihre  explanation  tor  the  workings  ot  this  bias  Is  that  it  mav  be  due 
to  a  leveling  effect.  Kssentlallv,  this  is  an  extrapolation  of  Simmel's 
concept  of  social  leveling  (Wolff,  I'lSO) .  If  a  group  of  individuals  are 
to  rate  each  other  on  how  well  each  performs-  his  lob,  vet  these  same 
individuals  perform  different  Jobs,  then  the  evaluation  must  be  based 
upon  the  lowest  level  of  Job  behaviors  common  to  all  members  v'f  the  group. 
Further,  If  the  original  promotion  svstem  Is  valid,  then  each  member  of 
a  given  rank  must  have  first  demonstrated  his  proficiency  at  each  lower 
rank,  the  best  persons  at  anv  given  rank  being  promoted  to  the  next 
higher  one.  Wljen  the  evaluation  is  comparative  across  ranks,  onlv  what 
Is  required  at  the  level  of  the  lowest  rank  found  in  the  group  is  a 
common  denominator  of  Job  behaviors  tor  all  ratees.  This  common  level 
ot  behavior  becomes  the  detlnltlon  of  "Job"  tor  the  evaluation  process. 
lUven  this  redefinition  of  "|ob"  awav  from  the  ratee's  actual  |ob  to  a 
lower  level  of  behaviors  common  to  all  ratees,  individuals  of  higher 
rank  are  seen  as  best  at  the  "  n'b"  and  those  of  the  lowest  ranks  are 
less  likely  to  be  seen  as  best  at  the  "lob." 

A  related  and  perhaps  more  plausible  explanation  is  that,  given  that 
individuals  at  different  ranks  perform  ditferent  Jobs,  the  raters  could 
not  be  expected  to  have  complete  knowledge  of  what  all  the  Jobs  entail. 

The  raters  must  then  rely  upon  social  indicators  of  proflclencv  on  the 
|ob,  redefining  "Job"  as  that  of  being  a  thiardsman.  The  definition  of 
Job  is  then  broad  enough  to  include  evervone  in  the  group  in  the  evaluation 
process.  In  this  case,  the  ratee's  rank  is  the  social  indicator  of 
proflclencv  at  being  a  Guardsman,  as  is  also,  to  a  lesser  extent,  length 
of  time  on  the  lob  (time  In  unit). 


b 


The  validity  of  either  of  the  above  explanatlona  as  to  how  tiu-  oiieot 
of  rank  operated  could  not  be  detemlned  fron  the  data.  Su.  h  .1  Hoionnl- 
natlon  would  require  an  analyala  of  the  underlying  process  hv  whioli  raters 
made  their  evaluations  and  nominations.  This  effect  poses  no  problem  wiien 
ratings  are  secured  during  basic  training  or  In  any  similar  situation  wtien 
the  group  members  are  performing  osaantlally  Identical  jobs,  liowev.-r,  in 
the  military  setting,  rank  by  definition  daalgnatss  unequals,  botit  in 
regard  to  formal  statue  and  Che  Job  to  bo  performed. 

The  Implication  to  ba  drawn  from  the  preaant  data  Is  iliat  onlv  tiiose 
of  equal  rank  are  by  definition  and  for  practical  purposes  equal .  an.i 
therefore  only  thaaa  are  true  poors.  Further,  the  peer  evaluation  process 
Itself  bocomos  Invalidated  to  the  extent  that  "pears"  are  not  defined  to 
Include  only  true  paers.  Ihidar  conditions  similar  to  those  of  the  present 
analysis  (chat  Is,  operational  unite).  It  would  be  preferable  to  define 
peer  groups  trlchin  sdlltary  rank.  In  any  situation  where  blogruphic 
characterlstlce  are  deemed  to  be  Independent  of  the  specific  evaluation 
purpose  yet  there  la  a  high  degree  of  relationship  between  the  two.  it 
Is  advisable  tomiae  ouch  biographic  Information  to  define  the  true  peor 
group  boundaries. 


SnMAltY 

'A  total  o>f  79  National  Cuardamen  took  part  In  research  to  analvze 
the  relationship  of  biographic  data  to  the  pear  evaluation  process. 
Significant  correlation  was  found  betwean  each  biographic  variable  and 
the  evaluation  score.  However,  partlalllng  rank  and  time  in  unit  on; 
of  these  relationships  reduced  the  correlation  to  nonsignificance,  in 
a  multiple  regression  analyola,  rank,  length  of  time  in  unit,  and 
education  accounted  for  SIX  of  Che  variance  In  pear  evaluation  scores. 

The  results  are  discussed  in  terms  of  lavallng  and  social  Indicator  et  feet 
which  nay  be  sources  of  bias  In  the  evaluation  of  pear  job  performance. 
Where  such  blaa  effects  are  found,  the  blas<-produclng  variables  should 
be  treated  as  delineators  of  true  peer  groups. 
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